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ABSTRACT 

This paper reports experiments involving the 
handling of flexible parts (e.g. wires) when 
using a teleoperated system with time delay. The 
task is principally a peg-in-hole task involving 
the wrapping of a wire around two posts on the 
task-board. It is difficult to estimate the effects 
of the flexible parts, therefore, on-line tele- 
operation is indispensable for this class of unpre- 
dictable task. 

We first propose a teleoperation system based 
on the predictive image display, then describe an 
experimental teleoperation testbed with a four- 
second transmission time delay. Finally, we 
report on wire handling operations that were 
performed to evaluate the performance of this 
system. Those experiments will contribute to 
future advanced experiments for the MIT1 ETS-VII 
mission. 

INTRODUCTION 

Remote manipulation in outer space from the 
ground is one of the most important technologies 
for assisting outer space activities such as the 
construction and maintenance of space stations, 
and the operation of space laboratories. The long 
distances between the ground command station 
and outer space robots incur an inevitable time 
delay of communications between these two 
systems; there are many research activities being 
conducted on this time delay problem. 
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Several ideas have been proposed: local intel- 
ligence with sensory feedback [1], a predictive 
image display system which superimposes a 
phantom robot with no delay on the remote 
camera image [2], a teleoperation system using 
force-reflecting simulator [3,4] and a tele- 
programming system which issues program 
segments to the remote site [5]. Space robot ex- 
periments have also been carried out on a space- 
lab mission [6]. 

In this paper we consider the tasks involved 
in handling flexible parts by a teleoperated 
system with communication delays, and focus on 
a wire handling task as an example of a general 
unpredictable task for teleoperation. 

This task is complicated for two reasons: 1) 
the dependence of the generated path on the 
changes of the shape of the flexible component, 
and 2) the difficulty of estimating the forces 
generated by the deformation of the flexible part. 
It is difficult to estimate the effects of the wire, 
and pre-programmed methods are not suitable 
for this class of task; an on-line teleoperation 
system is indispensable. 

We first propose a teleoperation system based on 
a predictive computer graphics display, then 
describe an experimental teleoperation testbed 
with a four-second communication time delay. 
Finally, we report on wire handling operations 
that were performed to evaluate the effectiveness 
of the system. 

TELE-OPERATION SYSTEM BASED 
ON PREDICTIVE CG DISPLAY 

In this section, we propose a remote manipu- 
lation system based on the predictive image 
display technique. Figure 1 shows the block 
structure of this system. The system consists of 
the master operating station subsystem and the 
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Figure 1. The proposed masler-slave tele-operation system 


slave arm subsystem, and these two are con- 
nected by a low-bandwidth communication line 
with a large time delay. 

If we assume a several-second-delay, we are 
not able to use the conventional master-slave 
system which consists of a large control loop, so 
we adopt the predictive graphic image display 
technique. 

The master subsystem is composed of a 
master handle, its controller, and a graphic simu- 
lator. This master subsystem simulates a virtual 
arm and displays it as a three-dimensional image. 
In this simulator, we ignore interactions with the 
environment and the hardware limitation of the 
slave system, hence the operator can control the 
virtual arm on the graphic display freely through 
the master handle. The series of configurations 
(position / orientation) of the virtual arm during 
operation are transmitted to the slave arm con- 
troller as a command configuration P c . 

The slave arm subsystem is composed of a 
slave arm and its double-loop controller which 
prevents the slave arm from excess loads. In the 
outer position loop, the operational force com- 
mand F c for the inner loop is given by 


F c = 


—F 

1 limit 

F 

limit 

K(P C -P) 


(in case K(P C -P)< -F^, ) 

(in case K(P C -P)> ) (1) 
(others) 


where F^-, is the limit force/torque to prevent 
the slave arm from applying excess loads, K is 
the gain parameter of the outer position loop, and 
P c and P are the commanded and the sensed 
configurations of the slave arm, respectively. 

We assume a position-controlled slave arm. 

In the inner force control loop, the reference 
position P r for the slave arm is calculated by 


using a nominal model of the slave arm. Its 
transfer function is as follows, 

F,(J)= . 1 - ( F c (s)-F(s )) (2) 

s(ms + b) 

where F is the sensed force at the tip of the slave 
arm, and m and b are the inertia and the damping 
parameters respectively of the nominal model. 
These parameters are designed to keep the 
bandwidth of the output reference position P r 
within the bandwidth of the slave arm. 

The operator manipulates the virtual arm on 
the graphic display using the master handle, 
sometimes watching the monitor of the slave arm 
system to check the motion of the slave arm for 
any failure of the wire-wrapping task. In the 
event of any such failure, the operator returns the 
virtual arm to its previous state and retries the 
wire wrapping. 

EXPERIMENTAL TELEOPERATION 
TESTBED 

To confirm the function of the proposed 
system, we constructed a teleoperation testbed 
with a four-second time delay. Figure 2 shows 
an overview of this experimental setup. The 
sequence of the command positions from the 
master subsystem is stored once in a ring buffer 
program which simulates a four-second com- 
munication time delay. The response of the slave 
arm is thus delayed by four seconds. 

An IRIS workstation (Crimson / Reality- 
Engine) is used for the three dimensional 
computer graphic display, and a newly designed 
hybrid master system (Fig. 3) is used for the 
master handle. This handle has three degrees of 
orientational freedom, and the orientation of the 
virtual arm follows the orientation of this master 
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Figure 2. Overview of the experimental setup 


handle. A six-axis force/torque sensor is 
installed at the base part of this handle, and the 
translational velocity of the virtual ami is propor- 
tional to the force which is applied by the 
operator. 

Two monitor displays are used for the master 
operating station, one to display the computer 
graphics which simulate the virtual arm, and the 
other to display the delayed camera image of the 
slave arm system. On the graphic display, the 
front view of the task board with two holes and 
two poles, and the manipulated peg, are 
displayed as 3D solid models. 

To check the real motion of the slave arm, the 
information of the real peg is also displayed 
super-imposed on this window as a 3D wire- 
frame model. On the top-right corner of the 
graphic display, a small window displays a side 
view of the slave arm system. On the top-left 
corner of the graphic display, another small 
window displays the force information of the 
slave arm. 

A direct-drive arm with six degrees of 
freedom is used for the slave arm. At the tip of 
this arm, a six-axis force / torque sensor is 
installed to detect the forces generated by the in- 
teraction with the environment. A slave arm 
control algorithm described before is imple- 
mented on a parallel processing system of trans- 
puters. The force limit Fu m u was set to 5N to 
protect the slave arm from excess loads during 
operation. 


As an example task of manipulating flexible 
parts, we tested a wire handling operation. We 
used a simple task-board with two holes, two 
poles and one manipulated peg with a thin copper 
cable. The clearance between the peg and the 
hole is 0.035mm. 

The task is principally a peg-in-hole task in- 
volving wrapping a wire around two posts on the 
task-board. The task consists of three stages; 
first, extracting the peg from the hole, second, 
wrapping the wire around the two poles, and 
third, inserting the peg into the initial hole. 

The results of the experiments are shown in 
Fig. 4 as a sequence of wire handling operations. 
Despite of the large communication time delay in 
this teleoperation system, we confirmed the 
success of the wire handling operation. 

To maintain consistency between the virtual 
arm space and the real slave arm space, we cali- 
brated the system prior to the experiments. 

CONCLUSION 

In this paper, we investigated a wire handling 
task as an example of an unpredictable task. We 
proposed a teleoperation system with the predic- 
tive image display and the double-loop slave 
controller, constructed a master-slave teleopera- 
tion testbed, and performed the wire handling 
task with a four-second communication delay. 





Figure 3. Structure of the hybrid master handle 
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MITI is planning to participate in the space 
robotic experiment on the ETS-VII [7], and an 
advanced robotic hand (ARH) with multiple 
degrees of freedom and sensors has been devel- 
oped for this mission [8]. This experiment will 
contribute to future advanced experiments for the 
MITI ETS-VII mission. 

This research was performed as a joint 
research project between the Mechanical 
Engineering Laboratory and the Research and 
Development Center of Toshiba Corporation. 
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Figure 4. Task sequence of a wire handling 
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